Skip to content

Conversation

sandhose
Copy link
Member

@sandhose sandhose commented Oct 31, 2024

On top of #3367

This does the bulk of processing jobs in the queue.

With this PR there are four main things missing from the job queue:

  • schedule jobs later (which we need to do exponential backoffs when retrying jobs)
  • schedule recurring jobs
  • reschedule jobs from dead workers
  • metrics

Copy link

cloudflare-workers-and-pages bot commented Oct 31, 2024

Deploying matrix-authentication-service-docs with  Cloudflare Pages  Cloudflare Pages

Latest commit: 6d92fb1
Status: ✅  Deploy successful!
Preview URL: https://2035bfc0.matrix-authentication-service-docs.pages.dev
Branch Preview URL: https://quenting-new-queue-jobs.matrix-authentication-service-docs.pages.dev

View logs

@sandhose sandhose force-pushed the quenting/new-queue/jobs branch from 6add451 to c455959 Compare October 31, 2024 17:14
@sandhose sandhose force-pushed the quenting/new-queue/insert-jobs branch from 28d01c0 to c3237ce Compare October 31, 2024 17:14
@sandhose sandhose force-pushed the quenting/new-queue/jobs branch from c455959 to 74e6eae Compare November 19, 2024 16:27
@sandhose sandhose force-pushed the quenting/new-queue/insert-jobs branch from c3237ce to 36921bd Compare November 19, 2024 16:27
@sandhose sandhose force-pushed the quenting/new-queue/jobs branch 5 times, most recently from 0d88685 to 86a6664 Compare November 21, 2024 10:19
@sandhose sandhose added the A-Jobs Related to asynchronous jobs label Nov 21, 2024
@sandhose sandhose marked this pull request as ready for review November 21, 2024 10:30
@sandhose sandhose requested a review from reivilibre November 22, 2024 15:53
@sandhose sandhose force-pushed the quenting/new-queue/insert-jobs branch from 36921bd to 4cbd066 Compare November 22, 2024 16:04
@sandhose sandhose force-pushed the quenting/new-queue/jobs branch from 86a6664 to 0d5c391 Compare November 22, 2024 16:04
Copy link
Contributor

@reivilibre reivilibre left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

generally LGTM

SELECT $1, queue_name, payload, metadata, $2, attempt + 1
FROM queue_jobs
WHERE queue_job_id = $3
AND status = 'failed'
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

should this also say AND next_attempt_id IS NULL?

@sandhose sandhose force-pushed the quenting/new-queue/insert-jobs branch from 4cbd066 to 0000132 Compare December 5, 2024 10:14
@sandhose sandhose force-pushed the quenting/new-queue/jobs branch from 0d5c391 to 954bd71 Compare December 5, 2024 10:14
@sandhose sandhose force-pushed the quenting/new-queue/insert-jobs branch from 0000132 to fdc3d9a Compare December 5, 2024 17:03
@sandhose sandhose force-pushed the quenting/new-queue/jobs branch from 954bd71 to 6d92fb1 Compare December 5, 2024 17:03
Base automatically changed from quenting/new-queue/insert-jobs to quenting/new-queue/merge December 6, 2024 08:22
@sandhose sandhose merged commit fcc99bc into quenting/new-queue/merge Dec 6, 2024
2 checks passed
@sandhose sandhose deleted the quenting/new-queue/jobs branch December 6, 2024 08:39
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

A-Jobs Related to asynchronous jobs

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants